#!/usr/bin/env python

# here's how maybe to make it reflective and tend to use its own words
# more often without adding a lot

# when it goes to respond to any phrase it preferentially saliency-checks
# phrases that it responded to. it does not choose a response of its own
# (even if it is not made up by the markov chainer, thus sensible) in the
# case that the next few speeches collected after he said it consitute
# negative reinforcement (emotionally negative, non-sequiturs, kicks/bans
# etc). positive phrases he'll preferentially choose and neutral ones
# (or ones that have been scored < confidencethreshold times) he'll
# also choose if the only other alternative is saliency-ranking the
# entire database of things other people were replying to.

# needed changes
# - tobmalf needs to break up 'lastspch' and put what he says in
#   the channel histories and uniquephrases if it's made up (this means
#   tobmalf's existing database needs to be massaged with something
#   like the new code, to consult his chatlog and insert his own speech
#   where it belongs, then do the assessment, which will be necessarily
#   fuzzy given the timer resolution in the chatlog. but future entries
#   will be non-fuzzy)
# - tobmalf must not treat lines ending with a question mark, starting
#   with a full colon or msg'd to him specially. these must all go to
#   the same handler
# for any of this to happen
# in so doing tobmalf fakes self-awareness and reflectiveness by
# heavily analyzing responses to things he says and preferring to
# quote himself before quoting another person or using the markov chainer
# (which tobmalf does now only when stumped). since tobmalf will be
# registering upon his sensorium and memory in all the same ways, but
# also much more deeply, as every other participant of his environment,
# and since those participants constitute that environment, at this
# point tobmalf will be fundamentally embodied and self-aware
# also, since people ought to respond well to stuff the markov chainer
# makes up which is sensible, likeable, funny or topical, he'll gradually
# gain a repertoire of well-scored things to say that he made up. however,
# the list of things he said he said will initially mostly consist of
# pointers to quotes of other people. phrases will contain names of 
# people not in the channel etc will be naturally pruned by this process
# and i'm hoping quoting people directly at all will be so disturbing
# on its own that invented phrases will come to outpopulate reused ones
# in the list of unique things tobmalf has said 
# alright

# when i am done tobmalf will be fully aware of his nature as an element of
# and primary actor of interest on his environment
#
# Joel Rosdahl <joel@rosdahl.net>
# Edited by Flamoot
# Edited by John Ohno <john.ohno@gmail.com>
# Edited and eviscerated by Mike Mol <mikemol@gmail.com>


# Mike's notes:
## I think most of the commands can probably be removed. I've removed some
## of them, and left mostly only ones I think are simple, useful and/or
## work. That means most of the pull-from-somewhere-else ones are gone.
## Those would be nice to have, but I want the code simplified to just
## the functionality that works. I left 'markov', as it produces interesting,
## albeit personally threatening, results.
#
## 'uniquesays' is the bot's conversation tracking. When he's looking for
## something to say, hey'll try pulling one of these lines, verbatim.
#
## 'recentsays' appears to be used to store a list of recent things he's
## said, which he tries to use to avoid repeating himself. Where it's
## updated, the code append()s the line he's saying, and then pop's the
## 0th element. It's a kind of round-robin approach, but implemented
## decidedly differently from how the logging round-robin buffer was/is.
## The logging round-robin buffer's size is tunable via a configuration
## opttion. 'recentsays' is statically initialized to a bunch of empty entries,
## the quantity of which is not configurable. This should probably change;
## being able to tune how frequently he's allowed to reuse sayings is very
## likely to have a significant impact on his originality and redundancy.
#
## I've found numerous apparent protections against hostile agents. For
## whatever reason these protections were needed, the code should probably
## be fixed so that the attacks wouldn't work to begin with. These are probably
## common-sense things like input sanitization. Analysis and refactoring of the
## code will hopefully reveal some of these types of issues.
#
## Changes I''d like to make (as I noted in #grlug 2/11/2011):
## * As far as parsing, I'd like to try to get him to detect parentheticals,
##   using commas and parens as bounds, and to detect compound sentences. I'd
##   like him to try to consume the relevant text segments individually, and
##   build his output using sentence construct patterns as he observes them.
## * I'd also like him to identify when a token matches the username of
##   someone in the channel, and replace that with a parseable/observable token
##   which might be replaced with someone else's name at a later time.
##   (Someone who'd recently interacted with him, for instance)
## * Identifying and stripping things like email addresses and URIs would be
##   another kind of token match-and-replace.
##     NOTE: URI stripping has been done in the markov routines.
#
## It'd also be good to have him ignore any line that began with a non-alpha-
## numeric character. Those tend to be control commands for other bots.
#
## Ok, it looks like this is how it works.
## 1) one of the on_[some]msg callbacks is hit.
## 2) on_[some]msg sets a global or parameter to getmessage, informing it
##    where to send its line.
## 3) getmessage buils a message, and passes it (along with the param?)
##    on to self.say.
## 4) self.say sees what the communication line the message is tagged with
##    is, and forms the message to pass onward.

from dotext import dotext
from tobutils import gzipin, gzipout, loadobj, log
from tobconf import saydelay, intrusionprob, threshold, port, channelz
from tobconf import nickname, server, securitykey, mutechans, NickServPass
from tobconf import NickServUser
from tobmarkov import markov, table

import os
import sys

from ircbot import SingleServerIRCBot
from irclib import nm_to_n, nm_to_h, irc_lower, ip_numstr_to_quad
from irclib import ip_quad_to_numstr

dotext("`$tobmalf is starting. aaaa")

import cPickle
import random
import glob
import time
import re

# chansums is a dictionary of lists of extractions ordered by channel
chansums = {}
chansumtimes = {}

# loads tobmalf.patterns-unique, stores it in 'uniquesays', global var
uniquesays = loadobj('tobmalf.patterns-unique', [], 'unique statements')

# loads 'tobmalf.personality',  Stores it in 'personality' (global var)
# Formerly had a direct usage of 'open', but nothing was saving to that
# format, so I killed it.
personality = loadobj('tobmalf.personality', {}, 'personality entries')

global_c_wtf = ""

# Don't-touch inits
recentsays = ["", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", "", ""]
#TODO: Only used in getmessage. Shouldn't need to be referenced outside there.
# Find what the deal is with this sentinel value.
lastspkr = {"#wtf-other-wtf":""}
#TODO: Only used in getmessage. Shouldn't need to be referenced outside there.
# Find what the deal is with this sentinel value.
lastspch = {"#wtf-other-wtf":""}

# A timestamp for when we last spoke. Used to stagger response times.
# Appears to be primarily for protection against flood protection.
# Should be tracked seperately per-server, and per-DCC connection,
# ultimately.
# Value contained is unix timestamp in 1/100s of a second.
lastsay = 0
  
# var name is apparently an acronym for "phrases scored since bot talked"
# Used in getmessage for 'emotional ranking' of responses.
pssbt = {}

# Used in getmessage for 'emotional ranking' of responses.
whobotsaw = {}
# Used in getmessage for 'emotional ranking' of responses.
whatbotsaw = {}

# loadsummaries() -- loads anemic predicates
def loadsummaries():
  global chansums
  global chansumtimes
  for fn in glob.glob('tobmalf.*.summary.anemic'):
    chan = fn.split('.summary')[0]
    chan = chan.split('tobmalf.')[1]
    mysumries = gzipin(fn)
    chansums.update({chan:mysumries[0]})
    chansumtimes.update({chan:mysumries[1]})
    sumcountstr = str(len(mysumries[0]))
    chandesc = (chan.replace('#', "#").replace(".", "."))
    dotext("`!loaded "+sumcountstr+" extractions for"+chandesc)

def get_srcid(server, tochan):
  """Converts the server and tochan arguments into a srcid"""
  return (server+"."+tochan).lower().replace('#', "")

def cleanLine(line):
  #TODO: Perhaps this should be done as a loop until no changes are discovered?
  # Also, it'd be nicer if it could handle arbitrary amounts of intervening
  # whitespace. We're probably talking regex, then.
  # Newlines to statements.
  line = line.replace("\n", ". ")
  
  # Repair (or create?) smileys
  for hint in [')', 'p', 'P', 'D', '(']:
    line = line.replace(". " + hint, ". :" + hint)
  
  # Strange ellipses and oddly-placed periods.
  line = line.replace(". .", ". ")
  line = line.replace(" .", ".")
  return line

def indent_string(level, active_levels, last):
  if level == 0:
    return ""
  s = ""
  for i in range(level-1):
    if i in active_levels:
      s = s + "| "
    else:
      s = s + "  "
  if last:
    s = s + "`-"
  else:
    s = s + "|-"
  return s

# Note that this apparently has nothing to do with sums,
# but perhaps with summaries?
def addtochansums(key, txt):
  global uniquesays
  global chansums
  global chansumtimes
  # Here, it looks like we're counting how many times exact phrases have been
  # used.
  key = key.lower().replace('#','')
  try:
    idx = uniquesays.index(txt)
  except:
    idx = -1
  # If .index(txt) threw an exception, we consider that we haven't seen this
  # phrase before.  At this point, we add the phrase to our memory, and find
  # its length.
  if idx == -1:
    idx = len(uniquesays)
    uniquesays.append(txt)
  # By this point, idx has been repurposed to refer to the length of the passed
  # phrase.  If the phrase is empty, we return out early.
  if idx == 0: # hmm
    print "idx 0 (got a blank), nothin to do."
    if not chansums.has_key(key):
      chansums.update({key:[idx]})
      # so things that expect us to always make a key don't break
      chansumtimes.update({key:[int(time.time())]}) 
    return
  # Increment our tracking of how recently we've seen this line.
  if not chansums.has_key(key):
    chansums.update({key:[idx]})
    chansumtimes.update({key:[int(time.time())]})
  else:
    chansums[key].append(idx)
    chansumtimes[key].append(int(time.time()))

class TestBot(SingleServerIRCBot):
  def __init__(self, channel, nickname, server, port=6667):
    SingleServerIRCBot.__init__(self, [(server, port)], nickname, nickname)
    self.channelz = channelz
    c = self.connection
    dotext("go go `%" + nickname)

  def on_nicknameinuse(self, c, e):
    c.nick(c.get_nickname() + "_")

  def on_welcome(self, c, e):
    global global_c_wtf
    markov("You keep on using that word. I do not think it means what you think it means.")
    global_c_wtf = c

    # Identify to services, if we can.
    if not None == NickServUser and not None == NickServPass:
      self.connection.privmsg(NickServUser, 'IDENTIFY shortcircuit ' + NickServPass)

    time.sleep(2)
    for channel in self.channelz:
      dotext("`0Joining `%" + channel)
      c.join(channel)
      time.sleep(1)

  def rejoin(self):
    global global_c_wtf
    for channel in self.channelz:
      global_c_wtf.join(channel)
      time.sleep(1)
    
  #TODO: Some other, saner authentication method needs to be employed.
  def on_privmsg(self, c, e):
    logmsg = "`4recv privmsg: tgt( "+e.target()+" ), src( "+e.source()+" ), "
    for arg in e.arguments():
      logmsg = logmsg + "arg( " + arg + "), "
    log(logmsg)

    print e.arguments()
    # have the key in your name to issue commands in msg.
    # Also, prepend your line with '!'
    speech = None
    if e.source().__contains__(securitykey) and e.arguments()[0][:1] == '!':
      cmd = e.arguments()[0][1:]
      self.do_command(e, cmd) 
    else:
      speech = self.getmessage('', e.source(), e.arguments()[0])
    
    if speech:
      # We have something to say.
      # Since this was a privmsg, we can send our message straight back to
      # its source. Since this is to an IRC network, we want to pause,
      # to avoid flooding.
      n = random.choice(['1', '1', '1', '5'])
      os.system("sleep "+n+"s")
      c.privmsg(e.source(), speech)

  def on_pubmsg(self, c, e):
    # Once in a while, randomly save ourself out.
    if int(random.random() * 500) == 5:
      dotext("`#[RANDOM DATAFILE DUMP. GET READY.]")
      self.writeout()
      dotext("`#[ALL DONE.]")
    global lastspch
    global chansums
    global chansumtimes
    global server
    # Log out the message we just saw.
    logmsg = "`5recv pubmsg: target( "+e.target()+" ), source( "+e.source()+" ), "
    for arg in e.arguments():
      logmsg = logmsg + "arg( " + arg + "), "
    log(logmsg)
    # I don't know what the \x02 is about, but it looks like we're sanitizing
    #  the input line and separating the addressee (x in "x: what's up?").
    a = e.arguments()[0].replace("\x02", "").split(":", 1)
    # Check if this is an "x: what's up"-formed phrase. If the part to the left
    # of the : would be valid as a username, strip it. If not, re-append the :
    # (clearly, it was there intentionally!) If there's no :, assign directly
    # to 'aone', which is a name that appears to repeatedly refer to the
    # message-to-be-consumed in various contexts.
    if len(a) > 1:
      argspaces = a[0].count(" ")
      nickpingcount = a[0].lower().count(nickname.lower())
      emptycheck = a[0].lower().count(nickname.lower().zfill(10))
      if argspaces == 0 and nickpingcount == 0 and emptycheck == 0:
        aone = a[1] # strip nicknames lines are addressed to unless it's the
                    # bot's name
      else:
        aone = a[0]+":"+a[1] # ._.
    else:
      aone = a[0] #stupid
    # e.target refers to where the message was sent. If it begins with a #, we
    # assume it's to an IRC channel.
    if e.target().startswith('#'):
      tochan = e.target()
    else:
      # I don't understand this. I think on_pubmsg is supposed to refer to
      # messages in open channels, but this suggests he received public
      # messages from channels he didn't know. (WALLOPS or similar, maybe?)
      # I've changed it to send to #dastoob, as that's tobmalf's central home
      # on Freenode.
      # UPDATE: That doesn't sem to accomplish anything. Just get rid of it
      # once we can pass the return value of getmessage to self.say().
      tochan = server+"."+"#dastoob" # pubmsg after all
    # If we should interpret this as a command (public commands? And wtf is
    # this with zfill(10)? Remind me to purge that before this source goes
    # public. Er...wait.
    # TODO: Tie in proper auth here.
    alen = len(a)
    astartswithbang = aone.startswith("!")
    loweraddr = irc_lower(a[0])
    #zfill(10) is the password for public commands ,_,
    pubkey = irc_lower(self.connection.get_nickname().zfill(10))
    pubauthed = loweraddr == pubkey
    if alen > 1 and astartswithbang or pubauthed:
      # Interperet as a command.
      self.do_command(e, aone.strip())
    else:
      # Not interpreted as a command. Munch on it and hopefully say something
      # interesting.
      msg = self.getmessage(tochan, e.source(), aone)
      # Send our reply
      if msg:
        # There are places we'd like to lurk and not speak.
        # Is this one of those places?
        lnick = tochan.lower()
        lowerlist = []
        for muted in mutechans:
          lowerlist.append(muted.lower())
        if lnick in lowerlist:
          dotext("`#Muted; not sending to " + tochan)
          return
        # Since this is to an IRC network, we want to wait a bit,
        # to reduce the chance of flooding.
        n = random.choice(['1', '1', '1', '5'])
        os.system("sleep "+n+"s")
        c.privmsg(e.target(), msg)

  def getmessageSpeakChoice(self,
                            tochan,
                            invoker,
                            aone,
                            intrusionprob,
                            nickname):
    alower = aone.lower()
    nicklower = nickname.lower()

    # TODO: Dependent on caller logic. Clean up.
    if not tochan:
      return True
    
    # Always respond if it's a private message.
    if tochan == "__"+invoker:
      return True

    # If the word 'everyone' was used, respond.
    if aone.count("everyone") > 0:
      return True

    # Based on random chance, speak.
    if random.choice(range(intrusionprob)) == 0:
      return True

    # If the word 'thanks' was used (as a separate word), speak.
    if aone.count(" thanks ")>0:
      return True

    # If the message began with 'any', speak.
    if alower.startswith("any"):
      return True

    # If the message contains our nick, speak.
    #TODO Find out what the deal with the ) replacement is.
    if alower.count(nicklower.replace(")", "")) > 0:
      return True

    # If we were greeted with 'hi' directly, speak.
    if alower.replace("_", "").replace(nickname, "").strip() == "hi":
      return True

    # If the word 'hello' was used, speak.
    if alower.count("hello") > 0:
      return True

    # Otherwise, return false.
    return False

  # TODO: break this up
  # NOTE! This thing has a _crapload_ of return points. What it sends is
  # _not_ its return value, but rather whenever and wherever it says
  # 'self.say'.
  def getmessage(self, tochan, source, msg):
    global lastspkr
    global lastspch
    global chansums
    global chansumtimes
    global server
    global personality
    global recentsays
    global uniquesays
    global pssbt
    global whobotsaw
    global whatbotsaw
    global threshold
    global intrusionprob

    aone = msg # ignore this, but know aone is the msg
    invoker = nm_to_n(source)
    
    # "__name" appears to be how it keeps track of direct conversations.
    # So, if it wasn't passed a 'tochan' value, it takes the invoker
    # (which was converted from the IRC-defined source) and tries to
    # build a workable target out of it.
    #
    # 'tochan' appears to only be provided by on_pubmsg. It's left empty
    # when called by on_privmsg or on_dccmsg. on_privmsg works.
    # on_dccmessage does not offer replies, so it seems likely that
    # this tochan logic is flawed wrt DCC CHAT conversations.
    # It would be useful to have a 'context' object passed for tochan,
    # which would marshal the reply. That would help toward genericizing
    # this method and allow it to be used more flexibly. (i.e. with
    # twitter, netcats, XMPP, etc.)
    if tochan == "":
      # oh let's try this
      tochan = "__"+invoker

    # Redundant 'if'! The above 'if' shoul never leave 'tochan' empty.
    # At worst, it might leave tochan as '__', if nm_to_n(source) came
    # back as an empty string. (Which was then stored in 'invoker')
    # I'd have to look at nm_to_n's source to figure out what conditions
    # that's possible under, if any.
    if tochan != "":
      # I don't know what this should do yet. I'd need to find out why
      # lastspkr might have keys. AFACT at this moment, it only holds a
      # string.
      srcid = get_srcid(server, tochan)
      if not lastspkr.has_key(srcid):
        lastspkr.update({srcid:""})
        mylastspkr = ""
      else:
        mylastspkr = lastspkr[srcid.lower().replace('#', "")]
      if not lastspch.has_key(srcid.lower().replace('#', "")):
        lastspch.update({srcid.lower().replace('#', ""):""})
        mylastspch = ""
      else:
        mylastspch = lastspch[(srcid).lower().replace('#', "")]
    else:
      # This code should never run; it's guarded against by the previous
      # if statement.
      mylastspkr = invoker
      mylastspch = aone
      
    if mylastspkr != invoker or tochan == "":
      msgsrc = server+"."+tochan
      srcid = get_srcid(server, tochan)

      # emotional ranking of responses near the bot's last speech in channel
      # history
      premoodadjustmentmls = mylastspch
      if pssbt.has_key(srcid):
        scoredSinceBotTalked = pssbt[srcid]
        if scoredSinceBotTalked < 4 and scoredSinceBotTalked > -1:
          scoredSinceBotTalked += 1
          if mylastspkr == whobotsaw[srcid] and scoredSinceBotTalked == 1:
            mylastspch = mylastspch.lstrip(whatbotsaw[srcid])
            
          if mylastspch != "":
            mylastspchmood = 0.0
          pssbt[srcid] = scoredSinceBotTalked
      mylastspch = premoodadjustmentmls

      if tochan != "":
        addtochansums(msgsrc, mylastspch)
        ext_count = str(len(chansums[srcid]))
        dotext (ext_count+" extractions accumulated for "+msgsrc)
      sys.stdout.flush()
      newmarkovlines = ""
      # Splits mylastspch into separate lines
      # Does some processing on each one:
      #   Removes leading and trailing whitespace
      #   Finds patterns like ".", ". ", ".  ", etc. to ". "
      #   Probably doesn't handle ".." or "..." well.
      # Adds "> msg me " to the front, "\n to the back
      # Concatenates it back as part of a new block of text.
      # So "blah blah\ngrah grah" becomes "> msg me blah blah>msg me grah grah"
      for hehhh in mylastspch.split("\n"):
        hehhh = hehhh.strip()
        hehhh = re.compile(r"(\.\s*){2,}", re.I).sub(". ", hehhh)
        # anything else we wanna cook irc text with
        if hehhh == "":
          continue
        newmarkovlines += "> msg me "+hehhh+"\n"
      if newmarkovlines != "":
        markov(newmarkovlines.strip())
      sys.stdout.flush()
      # markov() reads lines in log() format...
      #markovlines = "> msg me "+aone.strip()
      mylastspkr = invoker
      mylastspch = aone.strip()
    else:
      checkset = ['?', '!', '.', ',', ':', ';']
      if mylastspch and not any(mylastspch.endswith(sym) for sym in checkset):
        mylastspch += "."
      mylastspch += "\n"+aone.strip()
    if tochan != "":
      srcid = get_srcid(server, tochan)
      lastspkr[srcid] = mylastspkr
      lastspch[srcid] = mylastspch
    intrusiontries = 0
    maxintrusiontries = 5
    minintrusionsgenerated = 1 # this includes im a smart bot
    # this is a bad way to do this now, i should just find all the replies
    # to one highest-scored saliently related phrase at once instead of
    # repeatedly scoring all the phrases and finding a reply in a shuffled
    # list of channel summaries
    # tobmalf may try to respond to someone who speaks more than one line
    # (intrusion):
    global nickname

    if self.getmessageSpeakChoice(tochan,
                                  invoker,
                                  aone,
                                  intrusionprob,
                                  nickname):
      mylist = []      # important
      for achan in chansums:
        if achan != server+"."+tochan:
          mylist.append(achan)
        else:
          dotext("`#[skipping this channel]", sameline=1)

      tospeak = self.getMessageSpeak(mylastspkr,
                                     tochan,
                                     invoker,
                                     minintrusionsgenerated,
                                     intrusiontries,
                                     maxintrusiontries,
                                     mylastspch,
                                     mylist)

      # (Sometimes, tospeak will be None)
      if tospeak:
        oldtc = ""
        if tochan.startswith("__"): # in case of private msgs ,_,
          oldtc = tochan
          tochan = invoker

        # We have a bunch of linefeed-separated clauses.
        # Convert that to a single line, do some contraction.
        saymsg = " ".join(tospeak.split("\n"))
        saymsg = saymsg.replace("  ", " ")
        dotext("`3getmessage for " + tochan + " returning: " + saymsg)
        return saymsg
      else:
        dotext("`3getmessage for " + tochan + " came up empty.")
        
        # (FORMER) STATEMENT POINT
#        saydest = re.compile(r"^\#?__").sub('', tochan)
#        saymsg = " ".join(intrusion.split("\n"))
#        saymsg = saymsg.replace("  ", " ")
#        self.say(saydest, saymsg)
#        if oldtc.startswith("__"): # to be clean ,_,
#          tochan = oldtc

# end of getmsg
  def getMessageSpeak(self,
                      mylastspkr,
                      tochan,
                      invoker,
                      minintrusionsgenerated,
                      intrusiontries,
                      maxintrusiontries,
                      mylastspch,
                      mylist):
    #intrusions = []
    intrusions = [("im a smart bot", -1)]
    while len(intrusions) < minintrusionsgenerated:
      intrusiontries += 1
      if intrusiontries >= maxintrusiontries:
        dotext("\n`@[giving up on response]")
        break
        # indent from here
      likesToSay = random.choice(xrange(7))
      if likesToSay == 1 and len(personality) > 0:
        intrusions.append((random.choice(personality.keys()), -1))
        justsays = intrusions[-1][0]
        moodscore = str(personality[justsays])
        # major time saver
        dotext("`7just likes to say "+justsays+", mood score "+moodscore) 
        continue
      
      self.getMessagePerIntrusion(mylastspch, mylist, intrusions)

    tospeak = None

    if intrusions != []: #hm
      tospeak = self.getMessageRemainingIntrusions(intrusions,
                                                 tochan,
                                                 mylastspkr,
                                                 mylastspch,
                                                 invoker)

    return tospeak
    
    # End of GetMessageSpeak


  def getMessageRemainingIntrusions(self,
                                    intrusions,
                                    tochan,
                                    mylastspkr,
                                    mylastspch,
                                    invoker):
    print intrusions
    intrusions.remove(("im a smart bot", -1))

    # Note that this function is supposed to mod 'intrusions' and 'intrusion'
    intr = self.getMessageRecentSpeech(intrusions)
    intrusion = intr['intrusion']
    intrusionidx = intr['intrusionidx']


    if intrusion == "im a smart bot":
      intrusion = self.getMessageGenMarkovLine(mylastspch, tochan, mylastspkr, invoker)
      # Until we replace these 'im a smart bot' bits with None, just 
      # drop out early.
    else:
      recentsays.append(intrusion)
      recentsays.pop(0) # right
      srcid = get_srcid(server, tochan)
      whobotsaw.update({srcid:mylastspkr})
      whatbotsaw.update({srcid:mylastspch})
      pssbt.update({srcid:0})
      if not personality.has_key(intrusion):
        # that's enough on the markov side o_o
        personality.update({intrusion:0.0}) 

      if intrusion.count(",") > 0:
        if intrusion.split(",")[0].count(" ") == 0:
          intrusion = intrusion.split(",", 1)[1]
      if intrusion.startswith("//"):
        intrusion = "http:" + intrusion
      intrusion = intrusion.replace(" //", " http://")

    return intrusion


  def getMessageRecentSpeech(self, intrusions):
    againn = 1
    intrusion = 'im a smart bot'
    intrusionidx = -1
    while againn == 1:
      if intrusions != []:
        intrusiontuple = random.choice(intrusions)
        intrusion = intrusiontuple[0]
        intrusionidx = intrusiontuple[1]                    
        againn = 0
        if recentsays.count(intrusion) > 0:
          dotext ("`5skip '"+intrusion.replace("\n", " ")+"'; already said it.")
          intrusions.remove(intrusiontuple)
          againn = 1
        else:
          dotext("`5[intrusion #"+str(intrusionidx)+" picked ]")
      if intrusions == [("im a smart bot", -1)] or intrusions == []:
        intrusion = "im a smart bot"
        break
    return { 'intrusion':intrusion, 'intrusionidx':intrusionidx, }

  def getMessageGenMarkovLine(self, mylastspch, tochan, mylastspkr, invoker):
    if random.choice([1, 2, 3]) != 3:
      subseed = random.choice(mylastspch.replace("\n", " ").split(" "))
      senseed = ""
      lwrspch = mylastspch.lower()
      youcount = lwrspch.count(" you")
      mecount = lwrspch.count(nickname.lower().replace("_", ""))
      if youcount > 0 or mecount > 0:
        senseed = "I"
      elif mylastspch.lower().startswith("i "):
        senseed = "you"
      intrusion = ""
      while len(intrusion) < 1:
        # for privmsgs
        intrusion += markov("",
                            outputs=1,
                            w1="",
                            w2="",
                            subjectseed=subseed,
                            sentenceseed=senseed)

      recentsays.append(intrusion)
      recentsays.pop(0) # right
      srcid = get_srcid(server, tochan)
      whobotsaw.update({srcid:mylastspkr})
      whatbotsaw.update({srcid:mylastspch})
      pssbt.update({srcid:0})
      if not personality.has_key(intrusion):
        # that is enough on the markov side o_o
        personality.update({intrusion:0.0}) 

      if intrusion.strip().startswith("//"):
        intrusion = "http:"+intrusion.strip() # oops .-.

      return intrusion
      
  def getMessagePerIntrusion(self, mylastspch, mylist, intrusions):
    if mylastspch != "":
      random.shuffle(mylist)
      deadent = 0
      chanlogcount = str(len(mylist))
      correspondencecounter = 0
      cscstr = str(correspondencecounter)
      wtfctr = 0
      scores = {}
      wtfctr = 0
      woahwtfctr = 1
      highest = [-1, "", 0]
      while woahwtfctr < 6:
        for key in scores:
          if key[0] == wtfctr:
            if scores[(wtfctr, key[1], key[2])] > highest[0]:
              highest = [scores[(wtfctr, key[1], key[2])], key[1], key[2]]
        woahwtfctr = 7 # woah
      global threshold

      print "responding to '"+mylastspch+"'"

      if threshold > correspondencecounter:
        threshold == correspondencecounter - 1 # hmm
      if threshold < 1:
        threshold = 1
      if highest[0] > threshold: # threshold
        hghphrase = highest[1].replace("\n", " ")
        # less intrusive full colon massaging before putting text in the
        # phrases db. Now we gotta find a reply to this
        random.shuffle(mylist)
        foundphrase = ""
        foundidx = -1
        for somechan in mylist:
          somechannel = chansums[somechan]
          idx = 0
          while idx < len(somechannel):
            if somechannel[idx] == highest[2]:
              try:
                replyseekincrement = random.choice([1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 3])
                foundphrase = uniquesays[somechannel[idx+replyseekincrement]]
                foundidx = somechannel[idx+replyseekincrement]
              except:
                foundphrase = ""
              break
            idx += 1
          if foundphrase != "":
            break
        if foundphrase != "":
          tttt = cleanLine(foundphrase)
          myintrusion = (tttt, foundidx)
        else:
          tttt = cleanLine(highest[1])
          myintrusion = (tttt, highest[2])
        intrusions.append(myintrusion)
        # if use sets not lists ok
        intrusions = list(set(intrusions)) 
      else:
        dotext ("\n[phrase HS "+str(highest[0])+" not high enough, bailing]")
  # End of getMessagePerIntrusion


  def say(self, nick, msg, type="msg"):
    global lastsay
    global saydelay

    lnick = nick.lower()

    lowerlist = []
    for muted in mutechans:
      lowerlist.append(muted.lower())

    if lnick in lowerlist:
      dotext("`#Muted; not sending to " + nick)
      return

    log("`#send msg: nick( "+nick+" ), message( "+msg + " )")
      
    for line in msg.split("\n"):
      while int(time.time()*100) - lastsay < saydelay:
        time.sleep(.01)
        
      lastsay = int(time.time()*100)
    
      if type == "msg":
        try:
          self.connection.privmsg(nick, line)
        except:
          print "Error -- couldn't privmsg "+nick
      elif type == "notice":
        try:
          self.connection.notice(nick, line)
        except:
          print "Error -- couldn't notice "+nick


  def on_dccmsg(self, c, e):
    logmsg = "recv dccmsg: source( "+e.source()+" ), "
    for arg in e.arguments():
      logmsg = logmsg + "arg( " + arg + "), "
    log(logmsg)

    msg = self.getmessage('', e.source(), e.arguments()[0])
    c.privmsg(msg)
#     c.privmsg("You said: " + e.arguments()[0][:10]+" ...")


  def on_dccchat(self, c, e):
    if len(e.arguments()) != 2:
      return
    args = e.arguments()[1].split()
    if len(args) == 4:
      try:
        address = ip_numstr_to_quad(args[2])
        port = int(args[3])
      except ValueError:
        return
      self.dcc_connect(address, port)

  def nick_from_event(self, e):
    if e.target().startswith("#"):
      nick = e.target()
    else:
      nick = nm_to_n(e.source())
    return nick

  def writeout(self):
    global chansums
    global chansumtimes
    global server
    global uniquesays
    global personality
    dotext("[Writing out anemic channel summaries]")
    for k in chansums.keys():
      myk = k
      myfn = "tobmalf."+myk+".summary.anemic"
      gzipout(myfn, [chansums[k], chansumtimes[k]])
      dotext("[tobmalf."+myk+".summary.anemic written]")

    #use these somehow
    dotext("["+str(len(uniquesays))+" unique phrases saved.]")
    gzipout("tobmalf.patterns-unique", uniquesays)

    dotext("[writing markov table]")
    gzipout("tobmalf.markov.new", table)
    dotext("[tobmalf.markov written ("+str(len(table))+")]")

    dotext("[writing personality matrix]")
    gzipout("tobmalf.personality", personality)
    dotext("[tobmalf.personality written]")

  def do_command(self, e, cmd):
    global chansums
    global chansumtimes
    global global_c_wtf
    global intrusionprob
    global threshold
    
    # get out the name if the command starts with it   
    # NOTE THIS MAKES THE BOT NOT RESPOND WITH A NAME IF ADDRESSED WITH ITS
    # NAME BUT IT ALLOWS THE COMMANDS TO WORK. MAYBE SET A FLAG OR ONLY
    # TAKE OUT THE BOT'S NAME.ZFILL(10) INSTEAD OF .+?
    cmd = re.compile(r"^.+?: ").sub("", cmd) 
    
    dotext("Command received from "+e.source()+": "+cmd+"")
    
    #TODO: This should split on whitespace, not simply spaces.
    args = cmd.split(" ")
                       
    invoker = nm_to_n(e.source())
    
    if e.target().startswith("#"):
      nick = e.target()
    else:
      nick = nm_to_n(e.source())
        
    c = self.connection

    if cmd == "disconnect":
      self.disconnect()

    if cmd == "rejoin":
      self.say(nick, "rejoining channels")
      self.rejoin()

    elif cmd == "die":
      self.writeout()
      self.die()

    elif cmd.startswith("tell"):
      args = cmd.split(" ")
      if len(args) < 3:
        self.say(nick, "Usage: tell [#]foo baz baz baz ...")
      else:
        self.say(args[1], " ".join(args[2:]))
            
    elif cmd.startswith("intrusionprob"):
      args = cmd.split(" ")
      if len(args) < 2:
        reply = "Usage: !intrusionpron n"
        reply += " (where prob is 1/n)\nn ="
        reply += str(intrusionprob)
        self.say(nick, reply)
      else:
        try:
          intrusionprob = int(args[1])
        except:
          self.say(nick, "exception")

    elif cmd.startswith("threshold"):
      args = cmd.split(" ")
      global threshold
      if len(args) < 2:
        reply = "Usage: !threshold n"
        reply += " (where min object match is > n)\n"
        reply += "n = " + str(threshold)
        self.say(nick, reply)
      else:
        try:
          threshold = int(args[1])
        except:
          self.say(nick, "exception")

    elif cmd.startswith("writeout"):
      self.say(nick, "writing")
      dotext("[REQUESTED DATAFILE DUMP. GET READY.]")
      self.writeout()
      dotext("[ALL DONE.]")
      self.say(nick, "written")

# end of do_command



loadsummaries() # anemic now!

bot = TestBot(channelz, nickname, server, port)
bot.start()

